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Abstract: We establish consistency and asymptotic normality of the minimum 
density power divergence estimator under regularity conditions different from 
those originally provided by Basu et al. 



1. Introduction 

Basu et al. [l[ and 0] introduce the minimum density power divergence estimator 
(MDPDE) as a parametric estimator that balances infinitesimal robustness and 
asymptotic efficiency. The MDPDE depends on a tuning constant a > that con- 
trols this trade-off. For a = the MDPDE becomes the maximum likelihood estima- 
tor, which under certain regularity conditions is asymptotically efficient, see chapter 
6 of Lehmann and Casella In general, as a increases, the robustness (bounded 
influence function) of the MDPDE increases while its efficiency decreases. Basu et 
al. [l| provide sufficient regularity conditions for the consistency and asymptotic 
normality of the MDPDE. Unfortunately, these conditions are not general enough 
to establish the asymptotic behavior of the MDPDE in more general settings. Our 
objective in this article is to fill this gap. We do this by introducing new conditions 
for the analysis of the asymptotic behavior of the MDPDE. 

The rest of this note is organized as follows. In Section [2] we briefly describe 
the MDPDE. In Section [3] we present our main results for proving consistency 
and asymptotic normality of the MDPDE. Finally, in Section [4] we make some 
concluding comments. 



2. The MDPDE 

Let G be a distribution with support X and density g. Consider a parametric family 
of densities {f(x; 0) : 9 E 0} with x € X and C R p , p > 1. We assume this family 
is identifiable in the sense that if f(x;9i) = f(x;02) a.e. in x then Q\ = 02- The 
density power divergence (DPD) between an / in the family and g is defined as 

d*(9, f)= J x |/ 1+Q (x; 9)-(l+ g(x)f a (x; 9) + V+ Q (*) j dx 
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for positive a, and for a = as 

do(g,f) = lim \d a (g,f) = / 5(2;) log[fif(x)//(a;; 0)]dx. 

Note that when a = 1, the DPD becomes 

di (<?,/)= / [g(x)- f(x;9)] 2 dx. 
Jx 

Thus when a = the DPD is the Kullback-Leibler divergence, for a = 1 it is the 
L 2 metric, and for < a < 1 it is a smooth bridge between these two quantities. 
For a > fixed, we make the fundamental assumption that there exists a unique 
point 9q & O corresponding to the density / closest to g according to the DPD. The 
point #0 is defined as the target parameter. Let X±, . . . , X n be a random sample 
from G. The minimum density power estimator (MDPDE) of 9q is the point that 
minimizes the DPD between the probability mass function g n associated with the 
empirical distribution of the sample and /. Replacing g by g n in the definition of 
the DPD, d a (g, f), and eliminating terms that do not involve 9, the MDPDE 9 a ^ n 
is the value that minimizes 



/ f+-(x;9)dx-(l + -)-J2r(X. 
Jx V a J n fri 



=1 

over O. In this parametric framework the density /(•; #0) can be interpreted as the 
projection of the true density g on the parametric family. If, on the other hand, g 
is a member of the family then g — /(•; #0). 

Consider the score function and the information matrix of f(x;9), S(x;9) and 
i(x; 9), respectively. Define the p x p matrices K a {9) and J a {9) by 



(2.1) K a (9) = I S{x ] 9)S t {x;9)f Ia {x ] 9)g{x)Ax~U a {9)U t a {9), 

where 



x 

U a {9) - / S(x;9)f a (x;9)g(x)dx 
Jx 

and 

J a {9) = [ S{x;9)S t {x;9)f 1+a {x;9)dx 
Jx 

+ f {i(x; 9) - aS(x; 9)S t (x; 9)} x [g(x) - f(x; 9)}f a (x; 9)dx. 



(2.2) 



x 



Basu et al. [l| show that, under certain regularity conditions, there exists a se- 
quence 9 a:n of MDPDEs that is consistent for 9q and the asymptotic distribution of 
yn(9 a ,n — &o) is multivariate normal with mean vector zero and variance-covariancc 
matrix J a (9o)~ 1 K a (9o)J a (9o)^ 1 . The next section shows this result under assump- 
tions different from those of Basu et al. [l|. 

3. Asymptotic Behavior of the MDPDE 

Fix a > and define the function m : X x 9 — > R as 



(3.1) m{x,9)=[l + -)r(x-9)- ^f 1+a (x;9)dx 
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for all E 0. Then the MDPDE is an M- estimator with criterion function given by 
(|3.ip and it is obtained by maximizing 

1 n 

m„(6) = -J2m{Xi,9) 
n * — ' 

i=l 

over the parameter space 0. Let 0g Q be the set where 

(3.2) / \m(x,0)\g(x)dx < oo. 

Jx 

Clearly O G Qq, but we assume ©g has more points besides 0o- For 9 G ©g 
consider the expected value of m(X; 9) in (|3.ip under the true distribution G 

(3.3) M{9) =fl + V\J r(x; 9)g(x)dx ~ j f 1+a (x; 0)dx, 

and define M(9) = — oo for € 9 \ 0q. Then the target parameter 0q is such 
that — oo < M(0q) — sup 0e0 M{0) < oo. Furthermore, we assume that O may be 
endowed with a metric d. Heretofore it is assumed that (0, d) is compact. The next 
theorem establishes consistency of the MDPDE. 

Theorem 1. Suppose the following conditions hold. 

1. The target parameter 9q = argmaxA/e 6 e(^) exists and is unique. 

2. For € @q, i— > m(a;, 0) is upper semicontinuous a.e. in x. 

3. for sufficiently small balls B C Q, x <-* swp eeB m(x, 0) is measurable and 
satisfies 

/ sup m(x, 0)g(x)dx < oo. 
JxdeB 

Then any sequence of MDPDEs a ^ n that satisfies m n {9 a ^ n ) > m n (0o) — o p (l), is 
such that for any e > and every compact set K C 0, 

P(d(9 a>n> o ) >e,0 G K) -> 0. 

Proof. This is Theorem 5.14 of van der Vaart @ page 48. □ 

The first condition is our assumption of existence of 0$. It states that 0o is 
an element of the parameter space and it is unique (identifiable). Without this 
assumption there is no minimum density power estimation to do. Compactness of 
K is needed for {0 G K : d(0, 9q) > e} to be compact; this is a technical requirement 
to prove the theorem. If is not compact, one possibility is to compactify it. The 
third condition would follow if f(x; 0) is upper semicontinuous (trivially if it is 
continuous) in a.e. in x. Finally, the fourth condition is warranted by (|3.2[) in the 
interior of 0^. Thus we can claim the following result. 

Theorem 2. If condition 1 in Theorem [T] holds, and if f a (x;0) is upper semicon- 
tinuous (continuous) in in the interior of Qq and for a.e. in x, then any sequence 
9a,n of MDPDEs such that m n (0 a ^ n ) > m n (9o) — o p (l), satisfies d(0 atTl ,0o) — > 0. 

The asymptotic normality of the MDPDE hinges on smoothness conditions that 
are not required for consistency. These conditions are provided in the two following 
results. 
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Lemma 3. M{9) as given by (|3 . 3[) is twice continuous differentiable in a neigh- 
borhood B of 6*o with second derivative (Hessian matrix) Y\qM{6) = —(l + a)J a (9), 
if: 

1. The integral J x f l+a {x\9)dx is twice continuously differentiable with respect 
to 9 in B, and the derivative can be taken under the integral sign. 

2. The order of integration with respect to x and differentiation with respect to 
9 can be interchanged in M(9), for 9 G B. 

Proof. Consider the (transpose) score function S t (x; 9) = Dg log f(x; 9) and the in- 
formation matrix i(x;9) = —Hg\ogf(x;9) = —DgS(x;9). Also note that 
[Dgf(x; 9)]f a ~ 1 (x; 9) — S t (x;9)f a (x;9). Use the previous expressions and condi- 
tion 1 to obtain the first derivative of 9 <— > m(x; 9) 

(3.4) D e m(x,9) = (l + a)S t (x;9)f a {x;9)-(l + a) [ S\x; 9)f 1+a {x; 9)dx. 



Jx 

Proceeding in a similar way, the second derivative of 9 i— > m(x; 9) is 

H e m(x, 9) = (l+ a){-i{x; 9) + aS(x; 9)S t (x; 9)}f a (x; 9) - (1 + a) 

(3.5) , 

x I J -4{x; 9)f 1+a {x- 6) + (1 + a)S(x; 9)S t (x; 9)f 1+a {x- 9)dx 

Then using condition 2 we can compute the second derivative of M(9) under the 
integral sign and, after some algebra, obtain 

H 9 M{6) = J {H e m{x,9)}g{x)dx = -{l + a)J a {9). u 

The second result is an elementary fact about differentiable mappings. 

Proposition 4. Suppose the function 9 \— > m(x,9) is differentiable at 9q for x a.e. 
with derivative Dgm(x,9). Suppose there exists an open ball Be 9 and a constant 
M < oo such that \\ Dgm(x,9) ||< M for all 9 6 B, where \\ ■ \\ denotes the usual 
Euclidean norm. Then for every 9\ and 92 in B and a. e. in x, there exist a constant 
that may depend on x, 4>(x), such that 

(3.6) \m(x,9 1 )-m(x,9 2 )\<ct>(x) \\ 9 1 - 9 2 ||, 
and 

/ <f) 2 (x)g(x)dx < oo. 
Jx 

We can now establish the asymptotic normality of the MDPDE. 

Theorem 5. Let the target parameter 9q be an interior point of Q, and suppose the 
conditions of Lemma [3] and Proposition |4] hold. Then, any sequence of MDPDEs 
9 a ^ n that is consistent for 9q is such that 

Vn~(9 n , a - 9 ) ^ N p (0, J~ 1 (9o)K a {9 Q )J- 1 (9 Q )), 
where K a and J a are given in (|2.ip and (12. 2p . respectively. 
Proof. From Lemma M(9) admits the following expansion at #o 

M{6) = M(9 ) + \{6- 9o) t V a (9o)(9 - 9 ) + o(\\ 9 - 9 || 2 ), 

where V a (9) = HgM(9). Proposition |4] implies the Lipschitz condition (|3.6[) . Then 
the conclusion follows from Theorem 5.23 of van der Vaart 0]. □ 
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So far we have not given explicit conditions for the existence of the matrices 
J a and K a as defined by (|2.2| and (|2.1[) , respectively. In order to complete the 
asymptotic analysis of the MDPDE we now do that. Condition 2 in Lemma [3] 
implicitly assumes the existence of J a . This can be justified by observing that 
the condition that allows interchanging the order integration and differentiation in 
M(9) is equivalent to the existence of J a . For J a to exist we need ijk(x;9), the 
jfc-element of the information matrix i(x] 9), to be such that 

ijk{x; 9)f 1+a (x; 9)dx < oo, and / ijk(x; 9) f a (x; 6)g(x)dx < oo. 
i x Jx 

Regarding K a , let Sj(x;9) be the jth component of the score S(x;9). If 

(3.7) S 2 (x;9)f 2a (x;9)<C 3 , j = l,...,p, 

then 

S 2 (x;9)f 2a (x;9)g{x)dx < oo. 

Thus, K a exists. Furthermore, by (|3.4|) we see that the jth component of Dgm(x, 9) 
is 

Aj = {l + a)S j (x;9)f a (x;9)- (1 + a) [ S j (x;9)f 1+a (x;9)dx. 

Jx 

Then A 2 would be bounded by a constant Mj < oo if all the components Sj(x; 9) of 
the score vector S(x; 9) satisfy (|3.7[) . This is true because in this case Sj(x; 9)f a (x; 9) 
would be bounded by a constant too, and then 

S{x;9)f 1+a (x;9)dx < oo. 

Jx 
Hence 

p p 
|| D e m(x,9) \\ 2 = ^ A 2 <J2 M i < °°- 

i=l i=l 

Therefore, if p.7p holds, then the Lipschitz condition in (|3.6[) follows. 

From the previous analysis, we see that the conditions on M{9) and m(x, 9) given 
in Theorem [5] can be established in terms of the density f(x;9), its score vector 
S(x; 9), and its information matrix i(x; 9) as indicated in the next theorem. 

Theorem 6. The MDPDE is asymptotically normal, as in Theorem [5j if the fol- 
lowing conditions hold in a neighborhood B C Qq of 9$: 

1. Condition 1 of Lemma 3 

2. For each j = l,...,p, S 2 (x; 9)f 2a (x; 9) < Cj a.e. in x. 

3. Suppose there are functions 4>jk such that \ijk(x;9)f a (x;9)\ < 4>jk{x) for 
j,k = 1, . . . ,p, and 

(f>jk(x)f(x; 9)dx < oo, and / <pjk(x)g(x)dx < oo. 
x J x 



4. Concluding Remarks 



We have obtained consistency of the MDPDE under rather general conditions on 
the criterion function m. Namely, integrability of x <— > m(x, 9), and upper semicon- 
tinuity of 9 i— > m(x, ff). However these are not necessary conditions; concavity or 
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asymptotic concavity of m n (0) , would also give consistency of the MDPDE without 
requiring compactness of 0, see Giurcanu and Trindade [3j|. To decide which set of 
conditions are easier to verify seems to be more conveniently handled on a case by 
case basis. 
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